Personalized fitness activity training using augmented-reality based avatar

ABSTRACT

An electronic device and method for personalized fitness activity training using augmented-reality based avatar are provided. The electronic device receives a first set of images of a first user. The first set of images is captured for a duration in which the first user is engaged in a first fitness activity. The electronic device generates an augmented-reality display that includes a first avatar and an image of the first user based on the first set of images. The electronic device further controls a display device to render the generated augmented-reality display. The electronic device further determines posture information of the first user based on the first set of images. The electronic device determines real-time feedback based on application of a first neural network model on the determined posture information. The electronic device controls the first avatar to output the determined real-time feedback in the augmented-reality display.

CROSS-REFERENCE TO RELATED APPLICATIONS/INCORPORATION BY REFERENCE

None.

FIELD

Various embodiments of the disclosure relate to fitness activitytraining. More specifically, various embodiments of the disclosurerelate to an electronic device and method for personalized fitnessactivity training using augmented-reality based avatar.

BACKGROUND

With the busy schedule of individuals, there are many reasons that maycompel an individual to exercise (or workout) at home rather than goingto a gymnasium. Some of the benefits associated with exercising at homerather than the gymnasium includes eliminating the need to drive to thegymnasium, eliminating the need to comply with a set schedule (set bythe gymnasium), eliminating membership fee and other costs (such astravel costs), all within the comfort and safety of home. Existingfitness applications that exist in the market do not offer anyinteractivity or assessment, but include pre-loaded training videosrelated to different types of exercises. Typically, the individual mayview the training videos, and may imitate the trainer in these videos.However, due to lack of interactivity from such training videos, theindividual may not receive personalized and real-time feedback and/orrecommendations for improvement. Moreover, acquiring the services of apersonalized online trainer may be not only expensive but alsoinaccessible due to lack of trainers.

Limitations and disadvantages of conventional and traditional approacheswill become apparent to one of skill in the art, through comparison ofdescribed systems with some aspects of the present disclosure, as setforth in the remainder of the present application and with reference tothe drawings.

SUMMARY

An electronic device and a method for personalized fitness activitytraining using augmented-reality based avatar is provided substantiallyas shown in, and/or described in connection with, at least one of thefigures, as set forth more completely in the claims.

These and other features and advantages of the present disclosure may beappreciated from a review of the following detailed description of thepresent disclosure, along with the accompanying figures in which likereference numerals refer to like parts throughout.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram that illustrates an exemplary networkenvironment for personalized fitness activity training usingaugmented-reality based avatar, in accordance with an embodiment of thedisclosure.

FIG. 2 is a block diagram that illustrates an exemplary electronicdevice for personalized fitness activity training usingaugmented-reality based avatar, in accordance with an embodiment of thedisclosure.

FIGS. 3A-3C collectively illustrates exemplary operations forpersonalized fitness activity training using augmented-reality basedavatar, in accordance with an embodiment of the disclosure.

FIG. 4 is a diagram that illustrates exemplary operations for generationof a fitness routine and activity schedule for a first user, inaccordance with an embodiment of the disclosure.

FIG. 5 is a diagram that illustrates training of a first neural networkmodel to determine real-time feedback for a first fitness activity of afirst user, in accordance with an embodiment of the disclosure.

FIG. 6 is a first flowchart that illustrates exemplary operations forpersonalized fitness activity training using augmented-reality basedavatar, in accordance with an embodiment of the disclosure.

DETAILED DESCRIPTION

The following described implementations may be found in the disclosedelectronic device and method for personalized fitness activity trainingusing augmented-reality based avatar. The electronic device and methodmay render an augmented reality-based avatar that may providepersonalized assistance to the user to achieve a fitness goal of theuser. Exemplary aspects of the disclosure provide an electronic device(for example, a smart television, a laptop, or a mobile phone) that maybe configured to receive a set of images of a user. The set of imagesmay be captured for a duration in which the user is engaged in a fitnessactivity (such as aerobic, strength training, or Yoga). The electronicdevice may generate an augmented-reality display that includes an avatarand an image (for example, a live image) of the user based on the set ofimages. The electronic device may control a display device to render thegenerated augmented-reality display. The rendered augmented-realitydisplay may include the avatar configured to perform the fitnessactivity. The electronic device may determine posture information of theuser based on the set of images for the duration in which the user maybe engaged in the fitness activity. The electronic device may determinea real-time feedback based on an application of a neural network modelon the determined posture information. The determination of thereal-time feedback may be in response to performance of the fitnessactivity by the user. The electronic device may control the avatar tooutput the determined real-time feedback in the augmented-realitydisplay. The electronic device may thereby provide real-time andpersonalized feedback in the augmented-reality display based onassessment of posture of the user by the neural network model. Theavatar-based personalized assistance may be accessible any time of theday, while the augmented-reality display may simulate a personalizedtrainer experience in a real gymnasium environment. The electronicdevice may provide an interactive experience for the user during thefitness activity based on real-time feedback of the first avatar in anaugmented-reality environment.

The electronic device may utilize the neural network model and augmentedreality to provide personalized real-time feedback including improvementsuggestions to the user. In an embodiment, the electronic device mayoutput the real-time feedback as one of a movement of the first avatarin the augmented-reality display, a synthesized speech, or a textualfeedback. For example, in a case where the posture information indicatesthat the user is performing the fitness activity in a wrong manner (forexample, by employing a wrong posture), the electronic device maycontrol the first avatar to pause the fitness activity and demonstratethe correct posture for the fitness activity. In another example, in acase where the number of repetitions of the fitness activity by the useris less than a threshold, the electronic device may control the firstavatar to output a motivational phrase to motivate the user to finishthe repetitions. In an embodiment, the real-time feedback may beassociated with at least one of a movement of one or more parts of abody of the user, a posture of the user, the number of repetitions ofthe first fitness activity, the duration of the first fitness activity,or a breathing pattern of the user during the first fitness activity.

In an embodiment, the electronic device may be configured to receive auser input associated with at least one of a set of body parameters ofthe user, a user profile of the user, a fitness goal of the user, amedical condition of the user, or an experience level of the user inperforming the first fitness activity. The electronic device may furtheracquire a performance history of the user associated with one or moreprevious fitness activities of the user. The electronic device maygenerate a fitness routine that includes a suggestion of one or morepotential fitness activities based on at least one of the received userinput or the performance history. The electronic device may output thefitness routine on one of the display device or a user device associatedwith the user.

The electronic device may further determine, based on the fitnessroutine, a diet chart and an activity schedule for performing the firstfitness activity by the user. The electronic device may further outputone or more notifications periodically to the user device based on thedetermined diet chart and the activity schedule. The notifications mayinclude one of a reminder to perform the first fitness activity, areminder for consuming proper diet at proper intervals, or a status ofthe first fitness activity with respect to the fitness goal. Theelectronic device may periodically update the personalized fitnessroutine and activity schedule, which may provide insights about theprogress with respect to the fitness goal, improvement areas to achievethe fitness goal, and so forth. The electronic device may therebysimulate a personalized trainer experience based on the personalizedfitness routine, the personalized activity schedule, and the periodicnotifications.

FIG. 1 is a block diagram that illustrates an exemplary networkenvironment for personalized fitness activity training usingaugmented-reality based avatar, in accordance with an embodiment of thedisclosure. With reference to FIG. 1 , there is shown a networkenvironment 100. The network environment 100 may include an electronicdevice 102, a set of image sensors 104, a first neural network (NN)model 106, a display device 108, a server 110, and a communicationnetwork 112. The electronic device 102 may be communicatively coupled tothe set of image sensors 104, the first NN model 106, the display device108, and the server 110, via the communication network 112.

In FIG. 1 , the first NN model 106 is shown as being separate from theelectronic device 102. However, the disclosure may not be so limitingand in some embodiments, the first NN model 106 may be included in theelectronic device 102, without departing from scope of the disclosure.With reference to FIG. 1 , there is further shown an augmented-realitydisplay 114, a first set of images 116 of a first user 118, and a firstavatar 120. The first set of images 116 may include a first image 116A,a second image 116B up to an Nth image 116N.

The electronic device 102 may include suitable logic, circuitry,interfaces, and/or code that may be configured to receive the set ofimages 116 from the set of image sensors 104, generate theaugmented-reality display 114 including the first avatar 120, determineposture information of the first user 118 from the set of images 116,apply the first NN model 106 on the posture information, and control thefirst avatar 120 to provide real-time feedback to the first user 118 inthe augmented-reality display 114. The electronic device 102 maydownload an application related to the personalized fitness activitytraining from an application store/marketplace. The electronic device102 may execute the application to display a graphical user interfacefor the selection of the first fitness activity from a plurality offitness activities, the generation of the augmented-reality display 114,and the application of the first NN model 106 to provide the real-timefeedback. Examples of the electronic device 102 may include, but are notlimited to, a head-mounted display, an eXtended Reality (XR) device, awearable electronic device (such as smart glasses), a computing device,a personal computer, a computer workstation, a mainframe computer, ahandheld computer, a smartphone, a cellular phone, a gaming console, aserver, a smart television, and/or other computing devices withinformation processing and image processing capabilities.

Each of the set of image sensors 104 may include suitable logic,circuitry, and/or interfaces that may be configured to capture the firstset of images 116 (for example, a video) of the first user 118. In someembodiments, the set of image sensors 104 may be configured to capture asecond set of images of a set of users including the first user 118 anda second user. The set of image sensors 104 may be configured totransmit the captured first set of images 116 or the second set ofimages to the electronic device 102 in real time. The set of imagesensors 104 may include a single image sensor or multiple image sensorsconfigured to capture the first set of images 116 of the first user 118from one or more viewpoints. Examples of each of the set of imagesensors 104 may include, but are not limited to, a depth camera, awide-angle camera, an action camera, a closed-circuit television (CCTV)camera, a camcorder, a digital camera, camera phones, a time-of-flightcamera (ToF camera), a night-vision camera, and/or other image capturedevices. In FIG. 1 , the set of image sensors 104 is shown as beingseparate from the electronic device 102. However, the disclosure may notbe so limiting and in some embodiments, the set of image sensors 104 maybe integrated with the electronic device 102, without departing fromscope of the disclosure.

The first neural network (NN) model 106 may be a model that may betrained to accept a first set of key points associated with the firstuser 118 and to output various results in the form of a classificationresult associated with a posture of the first user 118. In anotherembodiment, the first NN model 106 may be configured to output variousresults in the form of recommendation results, clustering results,regression or prediction results, and/or a combination thereof.

The first NN model 106 (such as a convolutional neural network) may be amachine learning model, and may be defined by its hyper-parameters, forexample, activation function(s), number of weights, cost function,regularization function, input size, number of layers, and the like. Inan embodiment, the first NN model 106 may be a computational network ora system of artificial neurons or as nodes, arranged in a plurality oflayers. The plurality of layers of the first NN model 106 may include aninput layer, one or more hidden layers, and an output layer. Each layerof the plurality of layers may include one or more nodes (or artificialneurons, represented by circles, for example). Outputs of all nodes inthe input layer may be coupled to at least one node of hidden layer(s).Similarly, inputs of each hidden layer may be coupled to outputs of atleast one node in other layers of the first NN model 106. Outputs ofeach hidden layer may be coupled to inputs of at least one node in otherlayers of the first NN model 106. Node(s) in the final layer may receiveinputs from at least one hidden layer to output a result. The number oflayers and the number of nodes in each layer may be determined fromhyper-parameters of the first NN model 106. Such hyper-parameters may beset before training, while training, or after training the first NNmodel 106 on a training dataset.

Each node of the first NN model 106 may correspond to a mathematicalfunction (e.g., a sigmoid function or a rectified linear unit) with aset of parameters, tunable during training of the first NN model 106.The set of parameters may include, for example, a weight parameter, aregularization parameter, and the like. Each node may use themathematical function to compute an output based on one or more inputsfrom nodes in other layer(s) (e.g., previous layer(s)) of the first NNmodel 106. All or some of the nodes of the first NN model 106 maycorrespond to the same or a different mathematical function.

In accordance with an embodiment, the electronic device 102 may trainthe first NN model 106 on one or more features related to the set ofimages 116, one or more features related to the posture information ofthe first user 118, and so on, to obtain the trained first NN model 106.The first NN model 106 may be trained to classify the postureinformation of the first user 118 into good posture or bad posture, andto generate real-time feedback based on the performance of the fitnessactivity by the first user 118. For example, the electronic device 102may input the set of images 116 of the first user 118, a profile of thefirst user 118, sensor data associated with biological information ofthe first user 118, a set of key points associated with the postureinformation for each fitness activity, a movement of one or more bodyparts for each fitness activity, a number of repetitions for eachfitness activity, the duration of each fitness activity, a breathingpattern associated with each fitness activity, and so on, to train thefirst NN model 106.

In training of the first NN model 106, one or more parameters of eachnode of the first NN model 106 may be updated based on whether an outputof the final layer for a given input (from the training dataset) matchesa correct result based on a loss function for the first NN model 106.The above process may be repeated for the same or a different inputuntil a minima of loss function is achieved, and a training error isminimized. Several methods for training are known in art, for example,gradient descent, stochastic gradient descent, batch gradient descent,gradient boost, meta-heuristics, and the like.

The first NN model 106 may include electronic data, which may beimplemented as, for example, a software component of an applicationexecutable on the electronic device 102. The first NN model 106 may relyon libraries, external scripts, or other logic/instructions forexecution by a processing device, such as electronic device 102. Thefirst NN model 106 may include code and routines configured to enable acomputing device, such as the electronic device 102 to perform one ormore operations for classification of the posture information of thefirst user 118 and generation of the real-time feedback. Additionally,or alternatively, the first NN model 106 may be implemented usinghardware including, but not limited to, a processor, a microprocessor(e.g., to perform or control performance of one or more operations), afield-programmable gate array (FPGA), a co-processor, or anapplication-specific integrated circuit (ASIC). Alternatively, in someembodiments, the first NN model 106 may be implemented using acombination of hardware and software.

Examples of the first NN model 106 may include, but are not limited to,a Bayesian model, a decision tree, a Support Vector Machine, a deepneural network (DNN), a convolutional neural network (CNN), a recurrentneural network (RNN), an artificial neural network (ANN), a gatedrecurrent unit (GRU)-based RNN, a fully connected neural network, a deepBayesian neural network, a hybrid DNN, and/or a combination of suchnetworks.

The display device 108 may include suitable logic, circuitry, andinterfaces that may be configured to render the augmented-realitydisplay 114. In an embodiment, the augmented-reality display 114 on thedisplay device 108 may include the first avatar 120 and an image (forexample, a live image) of the first user 118. In an embodiment, theelectronic device 102 may determine a first position of the first user118 in the augmented-reality display 114 based on the first set ofimages 116. The electronic device 102 may control the display device 108to render the first avatar 120 at a second position in theaugmented-reality display 114 based on the determined first position.The electronic device 102 may control the display device 108 to displaythe augmented-reality display 114 so as to seamlessly blend the virtualworld (for example the first avatar 120 and its animations) and the realworld (for example, the first user 118 and background of the first user118). The display device 108 may be a touch screen which may enable auser to provide a user-input via the display device 108. The touchscreen may be at least one of a resistive touch screen, a capacitivetouch screen, or a thermal touch screen. The display device 108 may berealized through several known technologies such as, but not limited to,at least one of a Liquid Crystal Display (LCD) display, a Light EmittingDiode (LED) display, a plasma display, or an Organic LED (OLED) displaytechnology, or other display devices. In accordance with an embodiment,the display device 108 may refer to a display screen of a head mounteddevice (HMD), a smart-glass device, a see-through display, aprojection-based display, an electro-chromic display, or a transparentdisplay. In FIG. 1 , the display device 108 is shown as being separatefrom the electronic device 102. However, the disclosure may not be solimiting and in some embodiments, the display device 108 may beintegrated with the electronic device 102, without departing from scopeof the disclosure.

The server 110 may include suitable logic, circuitry, interfaces, and/orcode that may be configured to store user-specific data, such as but notlimited to, a user profile of the first user 118, a set of bodyparameters of the first user 118, a fitness goal of the first user 118,a medical condition of the first user 118, an experience level of thefirst user 118, a performance history associated with one or moreprevious fitness activities of the first user 118, a progress of thefirst user 118 with respect to the fitness goal, the fitness routine ofthe first user 118, the activity schedule of the first user 118, thediet plan of the first user 118, and so on. In some embodiments, theserver 110 may store metadata for generation of a plurality of avatarsincluding the first avatar 120, metadata (such as number of repetitionsper level, variations, tempo) associated with a plurality of fitnessactivities, posture information (such as good and bad postures)associated with each fitness activity, a plurality of animations of thefirst avatar 120 corresponding to each fitness activity, a plurality ofanimations corresponding to output of the real-time feedback by thefirst avatar 120 on the display device 108, and so on. In anotherembodiment, the server 110 may be configured to train the first NN model106 and a second NN model (shown in FIG. 2 ), and transmit the trainedfirst NN model 106 and the trained second NN model to the electronicdevice 102. The second NN model (for example, OpenPose model) may betrained for determination of the posture information of the first user118 based on the first set of images 116, and for outputting a pluralityof key points corresponding to joints of a body of the first user 118.The server 110 may be configured to store the training dataset fortraining the first NN model 106 and the second NN model, and may updatethe training data set periodically.

In an embodiment, the server 110 may be implemented as a cloud serverwhich may execute operations through web applications, cloudapplications, HTTP requests, repository operations, file transfer, andthe like. Other examples of the server 110 may include, but are notlimited to a database server, a file server, a web server, a mediaserver, an application server, a mainframe server, a cloud server, orother types of servers. In one or more embodiments, the server 110 maybe implemented as a plurality of distributed cloud-based resourcesutilizing several technologies that are well known to those skilled inthe art. A person of ordinary skill in the art will understand that thescope of the disclosure may not be limited to implementation of theserver 110 and the electronic device 102 as separate entities. Incertain embodiments, the functionalities of the server 110 may beincorporated in its entirety or at least partially in the electronicdevice 102, without departing from the scope of the disclosure.

The communication network 112 may include a communication medium throughwhich the electronic device 102, the first NN model 106, and the server110 may communicate with each other. The communication network 112 maybe a wired or wireless communication network. Examples of thecommunication network 112 may include, but are not limited to, theInternet, a cloud network, a Wireless Fidelity (Wi-Fi) network, aPersonal Area Network (PAN), a Local Area Network (LAN), or aMetropolitan Area Network (MAN). Various devices in the networkenvironment 100 may be configured to connect to the communicationnetwork 112, in accordance with various wired and wireless communicationprotocols. Examples of such wired and wireless communication protocolsmay include, but are not limited to, at least one of a TransmissionControl Protocol and Internet Protocol (TCP/IP), User Datagram Protocol(UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP),Zig Bee, EDGE, IEEE 802.11, light fidelity (Li-Fi), 802.16, IEEE802.11s, IEEE 802.11g, multi-hop communication, wireless access point(AP), device to device communication, cellular communication protocols,and Bluetooth (BT) communication protocols.

In operation, the electronic device 102 may receive a user input forselection of the first fitness activity from a set of fitnessactivities. The user input may indicate that the first user 118 may beready to perform the first fitness activity. The first fitness activitymay correspond to an exercise/workout to improve fitness and wellbeingof the first user 118. For example, the first fitness activity maycorrespond to an exercise related to a specific part of a body of thefirst user 118. For example, the exercises related to the chest of thefirst user 118 may include, but are not limited to, bench press,dumbbell press, dumbbell pullover, dumbbell fly, decline press, inclinedumbbell fly, incline press, incline dumbbell fly, and the like.Similarly, the exercises related to triceps of the first user 118 mayinclude, but are not limited to, triceps pushdown, seated barbellextension, overhead triceps extension, overhead barbell extension,one-arm dumbbell extension, seated dumbbell extension, dumbbellkickback, dumbbell triceps extension, and the like. In some embodiments,the first fitness activity may include aerobic exercises, Yoga, danceforms, running, gymnastics, and the like.

Based on the reception of the user input, the electronic device 102 maybe configured to control the set of image sensors 104 to capture thefirst set of images 116 of the first user 118. The first set of images116 may be captured for a duration in which the first user 118 may beengaged in the first fitness activity. The electronic device 102 may befurther configured to generate the augmented-reality display 114 thatmay include the first avatar 120 and an image (for example, a liveimage) of the first user 118 based on the first set of images 116. In anembodiment, the electronic device 102 may determine a first position ofthe first user 118 in the augmented-reality display 114 based on thecaptured first set of images 116. The electronic device 102 control thedisplay device 108 to render the first avatar 120 in theaugmented-reality display 114 at a second position based on thedetermined first position. For example, the second position of the firstavatar 120 in the augmented-reality display 114 may be adjacent to andspaced apart from the first position of the first user 118, as shown inFIG. 1 . The position of the first avatar 120 may not be limited to thatshown in FIG. 1 , and may include other positions so long as the firstavatar 120 is clearly distinguishable from the image of the first user118 in the augmented-reality display 114. The electronic device 102 maybe further configured to control the display device 108 to render thegenerated augmented-reality display 114.

The electronic device 102 may then control the first avatar 120 to begindemonstration of the selected first activity on the display device 108.In an embodiment, the electronic device 102 may detect a specificgesture of the first user 118 based on the first set of images 116 totrigger the first avatar 120 to start the first fitness activity. Forexample, the electronic device 102 may detect a finger snapping gestureor a finger rolling gesture to trigger the first avatar 120 to start thefirst fitness activity. In another embodiment, the electronic device 102may detect a specific input on an interface associated with theelectronic device 102 to trigger the first avatar 120 to start the firstfitness activity. For example, the specific input may comprise pressingof a specific button on the remote control of a smart television,pressing of a specific button on a handheld controller of a gamingconsole, or pressing of a specific button on a graphical user interfaceof a smart phone. The electronic device 102 may detect whether the firstuser 118 has started to imitate or copy the first avatar 120 to performthe first fitness activity. In a case where the electronic device 102detects that the first user 118 has not started performing the firstfitness activity, the electronic device 102 device may pause the firstavatar 120, and verbally or visually prompt the first user 118 torestart the first fitness activity using the specific gesture or thespecific input. In a case where the electronic device 102 detects thatthe first user 118 has started performing the first fitness activity,the electronic device 102 device may control the first avatar 120 tocontinue the demonstration of the first fitness activity.

The electronic device 102 may be further configured to determine postureinformation of the first user 118 based on the first set of images 116for the duration in which the first user 118 is engaged in the firstfitness activity. The posture information may include a plurality of keypoints corresponding to joints and/or parts of a body of the first user118. The posture information may indicate a posture of the first user118 employed during the performance of the first fitness activity. Thedetails about determination of the posture information are provided, forexample, in FIGS. 3A and 3B.

Based on the determination of the posture information, the electronicdevice 102 may be further configured apply the first NN model 106 of aset of neural networks on the determined posture information todetermine real-time feedback. The determination of the real-timefeedback may be in response to performance of the first fitness activityby the first user 118. The determined real-time feedback may beassociated with at least one of a movement of one or more parts of thebody of the first user 118, the posture of the first user 118, a numberof repetitions of the first fitness activity, the duration of the firstfitness activity, a breathing pattern of the first user 118 during thefirst fitness activity, and so on. In an embodiment, the determinedreal-time feedback may include one or more improvement suggestions basedon the performance of the first user 118. In another embodiment, thedetermined real-time feedback may include a motivational phrase based onthe performance of the first user 118.

The electronic device 102 may further control the first avatar 120 tooutput the determined real-time feedback in the augmented-realitydisplay 114. For example, the electronic device 102 may be furtherconfigured to output the determined real-time feedback as one of amovement of the first avatar 120 in the augmented-reality display 114, asynthesized speech, or a textual feedback. The details of determinationand output of the real-time feedback are provided, for example, in FIGS.3A and 3B. The electronic device 102 may thereby provide real-time andpersonalized feedback in the augmented-reality display 114 based onassessment of the posture of the first user 118 by the first NN model106. The electronic device 102 may provide an interactive experience forthe first user 118 and may simulate a personalized trainer experienceduring the first fitness activity based on real-time feedback of thefirst avatar 120 in an augmented-reality environment.

For example, in a case where the posture information indicates that thefirst user 118 is performing the first fitness activity in a wrongmanner (for example, by employing a wrong posture), the electronicdevice 102 may control the first avatar 120 to pause the first fitnessactivity and may demonstrate the correct posture for the fitnessactivity using other animations (such as other views or closeup views).The electronic device 102 may also output verbal feedback (such asfeedback on posture, feedback on tempo, feedback on breathing pattern,and so on). In another example, in a case where the number ofrepetitions of the first fitness activity by the first user 118 is lessthan a threshold, the electronic device 102 may control the first avatar120 to output a motivational phrase to motivate the user to finish therepetitions.

In an embodiment, the electronic device 102 may be configured to receivea user input associated with at least one of a set of body parameters ofthe first user 118, a user profile of the first user 118, a fitness goalof the first user 118, a medical condition of the first user 118, or anexperience level of the first user 118 in performing the first fitnessactivity. The electronic device 102 may further acquire a performancehistory of the first user 118 associated with one or more previousfitness activities of the first user 118. The electronic device 102 maygenerate a fitness routine that includes a suggestion of one or morepotential fitness activities based on at least one of the received userinput or the performance history. The electronic device 102 may outputthe fitness routine on one of the display device 108 or a user device(for example, a smart phone) associated with the first user 118.

The electronic device 102 may further determine, based on the fitnessroutine, a diet chart and an activity schedule for performing the firstfitness activity by the first user 118. The electronic device 102 mayfurther output one or more notifications periodically to the user device(for example, the smart phone) based on the determined diet chart andthe activity schedule. The notifications may include one of a reminderto perform the first fitness activity, a reminder for consuming properdiet at proper intervals, or a status of the first fitness activity withrespect to the fitness goal. The electronic device 102 may periodicallyupdate the personalized fitness routine and the activity schedule, whichmay provide insights about the progress of the first user 118 withrespect to the fitness goal, improvement areas to achieve the fitnessgoal, and so forth. The electronic device 102 may thereby simulate apersonalized trainer experience based on the personalized fitnessroutine, the personalized activity schedule, and the periodicnotifications.

Modifications, additions, or omissions may be made to FIG. 1 withoutdeparting from the scope of the present disclosure. For example, thenetwork environment 100 may include more or fewer elements than thoseillustrated and described in the present disclosure. For instance, insome embodiments, the network environment 100 may include the electronicdevice 102 and an audio rendering device. As another instance, the setof image sensors 104 and the display device 108 are shown as separatefrom the electronic device 102. However, the disclosure may not be solimiting and in some embodiments, the set of image sensors 104 and thedisplay device 108 may be integrated with the electronic device 102,without departing from scope of the disclosure.

FIG. 2 is a block diagram that illustrates an exemplary electronicdevice for personalized fitness activity training usingaugmented-reality based avatar, in accordance with an embodiment of thedisclosure. FIG. 2 is explained in conjunction with elements from FIG. 1. With reference to FIG. 2 , there is shown a block diagram 200 of theelectronic device 102. The electronic device 102 may include circuitry202 which may perform operations for personalized fitness activitytraining using augmented-reality based avatar. The electronic device 102may further include a memory 204, an input/output (I/O) device 206, anda network interface 208. The memory 204 may include the first NN model106 and a second neural network (NN) model 210. The circuitry 202 may becommunicatively coupled to the memory 204, the I/O device 206, and thenetwork interface 208.

The circuitry 202 may include suitable logic, circuitry, and interfacesthat may be configured to execute program instructions associated withdifferent operations to be executed by the electronic device 102. Forexample, some of the operations may include reception of the first setof images 116, generation of the augmented-reality display 114, controlof the display device 108 to display the augmented-reality display 114,determination of the posture information, determination of the real-timefeedback, and control of the first avatar 120 to output the determinedreal-time feedback. The circuitry 202 may include one or morespecialized processing units, which may be implemented as a separateprocessor. In an embodiment, the one or more specialized processingunits may be implemented as an integrated processor or a cluster ofprocessors that perform the functions of the one or more specializedprocessing units, collectively. The circuitry 202 may be implementedbased on a number of processor technologies known in the art. Examplesof implementations of the circuitry 202 may be an x86-based processor, aGraphics Processing Unit (GPU), a Reduced Instruction Set Computing(RISC) processor, an Application-Specific Integrated Circuit (ASIC)processor, a Complex Instruction Set Computing (CISC) processor, amicrocontroller, a central processing unit (CPU), and/or other controlcircuits.

The memory 204 may comprise suitable logic, circuitry, interfaces,and/or code that may be configured to store the instructions to beexecuted by the circuitry 202. The memory 204 may be configured to storethe received first set of images 116, a plurality of key points of ahuman body, the classification result, an activity schedule for thefirst user 118, a diet plan for the first user 118, and captured sensordata related to biological information of the first user 118. In someembodiments, the memory 204 may be configured to store user-specificdata, such as, a set of body parameters of the first user 118, a userprofile of the first user 118, a fitness goal of the first user 118, amedical condition of the first user 118, or an experience level of thefirst user 118 in performing the first fitness activity. In someembodiments, the memory 204 may download from the server 110 and storemetadata for generation of a plurality of avatars including the firstavatar 120, metadata (such as number of repetitions per level,variations, tempo) associated with a plurality of fitness activities,posture information (such as good and bad postures) associated with eachfitness activity, a plurality of animations of the first avatar 120corresponding to each fitness activity, a plurality of animationscorresponding to output of the real-time feedback by the first avatar120, and so on. The memory 204 may be further configured to store thefirst NN model 106 and the second NN model 210, and the training datasetfor both the first NN model 106 and the second NN model 210. The memory204 may be further configured to store several instances of thereal-time feedback and the progress of the first user 118 with respectto the fitness goal over a period of time. Examples of implementation ofthe memory 204 may include, but are not limited to, Random Access Memory(RAM), Read Only Memory (ROM), Electrically Erasable ProgrammableRead-Only Memory (EEPROM), Hard Disk Drive (HDD), a Solid-State Drive(SSD), a CPU cache, and/or a Secure Digital (SD) card.

The I/O device 206 may include suitable logic, circuitry, and interfacesthat may be configured to receive the user input(s) and provide anoutput based on the received user input(s). The I/O device 206 may beconfigured to receive the user input for selection of the first fitnessactivity or a second fitness activity from a set of fitness activities.The I/O device 206 may be configured to receive the user inputassociated with at least one of the set of body parameters of the firstuser 118, the user profile of the first user 118, the fitness goal ofthe first user 118, the medical condition of the first user 118, or theexperience level of the first user 118 in performing the first fitnessactivity. The I/O device 206 may be configured to receive the user inputto trigger the first avatar 120 to start the first fitness activity. TheI/O device 206 may be configured to control the first avatar 120 tooutput the determined real-time feedback in the augmented-realitydisplay 114. The I/O device 206 which may include various input andoutput devices, which may be configured to communicate with thecircuitry 202. Examples of the I/O device 206 may include, but are notlimited to, the display device 108, an audio rendering device, a touchscreen, a keyboard, a mouse, a handheld controller, a radio wavetransceiver, an infrared transceiver, a joystick, and a microphone.

The network interface 208 may include suitable logic, circuitry, andinterfaces that may be configured to facilitate communication betweenthe circuitry 202, the set of image sensors 104, the first NN model 106,the display device 108, and the server 110, either directly or via thecommunication network 112. The network interface 208 may be implementedby use of various known technologies to support wired or wirelesscommunication of the electronic device 102 with the communicationnetwork 112. The network interface 208 may include, but is not limitedto, an antenna, a radio frequency (RF) transceiver, one or moreamplifiers, a tuner, one or more oscillators, a digital signalprocessor, a coder-decoder (CODEC) chipset, a subscriber identity module(SIM) card, or a local buffer circuitry. The network interface 208 maybe configured to communicate via wireless communication with networks,such as the Internet, an Intranet or a wireless network, such as acellular telephone network, a wireless local area network (LAN), and ametropolitan area network (MAN). The wireless communication may beconfigured to use one or more of a plurality of communication standards,protocols and technologies, such as Global System for MobileCommunications (GSM), Enhanced Data GSM Environment (EDGE), widebandcode division multiple access (W-CDMA), Long Term Evolution (LTE), codedivision multiple access (CDMA), time division multiple access (TDMA),Bluetooth, Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE802.11b, IEEE 802.11g or IEEE 802.11n), voice over Internet Protocol(VoIP), light fidelity (Li-Fi), Worldwide Interoperability for MicrowaveAccess (Wi-MAX), a protocol for email, instant messaging, and a ShortMessage Service (SMS).

The second NN model 210 may be a computational network or a system ofartificial neurons, arranged in a plurality of layers, as nodes.Examples of the second NN model 210 may include, but are not limited to,a deep neural network (DNN), a convolutional neural network (CNN), afully connected neural network, and/or a combination of such networks.The second NN model 210 may be similar in functionality andconfiguration to the first NN model 106 shown in FIG. 1 ; however thesecond NN model 210 may differ from the first NN model 106 in terms ofthe training dataset used to train the second NN model 210, and theexpected output of the second NN model 210. Accordingly, the detaileddescription of the second NN model 210 is omitted herein, for the sakeof brevity. In accordance with an embodiment, the electronic device 102may train the second NN model 210 on one or more features related to theset of images 116, one or more features related to joints and/or partsof the body of the first user 118, and so on, to obtain the trainedsecond NN model 210. The second NN model 210 may be trained to estimatethe posture information of the first user 118, and to generate aplurality of key points corresponding to the parts (such as elbow,wrist, shoulder, neck, head, eyes, hip, knee, ankle, etc.) of the bodyof the first user 118. The second NN model may correspond to, forexample, Open Pose model or any other pose estimation algorithm.

The functions or operations executed by the electronic device 102, asdescribed in FIG. 1 , may be performed by the circuitry 202. Operationsexecuted by the circuitry 202 are described in detail, for example, inFIGS. 3-6 .

FIG. 3A illustrates exemplary operations for personalized fitnessactivity training using augmented-reality based avatar, in accordancewith an embodiment of the disclosure. FIG. 3A is explained inconjunction with elements from FIGS. 1 and 2 . With reference to FIG.3A, there is shown a block diagram 300A that illustrates exemplaryoperations from 302A to 302F, as described herein. The exemplaryoperations illustrated in the block diagram 300A may start at 302A andmay be performed by any computing system, apparatus, or device, such asby the electronic device 102 of FIG. 1 or the circuitry 202 of FIG. 2 .Although illustrated with discrete blocks, the exemplary operationsassociated with one or more blocks of the block diagram 300A may bedivided into additional blocks, combined into fewer blocks, oreliminated, depending on the particular implementation.

At 302A, a user profile of the first user 118 may be created. In anembodiment, the circuitry 202 may be configured to create a user profileof the first user 118. To create the user profile, the circuitry 202 mayreceive an initial user input 304 associated with a set of bodyparameters of the first user 118. The set of body parameters of thefirst user 118 may include an age of the first user 118, a height of thefirst user 118, a weight of the first user 118, a body mass index of thefirst user 118, a gender of the first user 118, a waist size of thefirst user 118, a size of arms of the first user 118, a chest size ofthe first user 118, and the like.

In another embodiment, the received initial user input 304 may befurther associated with at least one of a fitness goal (for example,weight loss, weight gain, stamina, etc.) of the first user 118, amedical condition of the first user 118, an experience level of thefirst user 118 in performing the first fitness activity, and so on. Thefitness goal of the first user 118 may include a time period forachieving the fitness goal. For example, the fitness goal of the firstuser 118 may be losing a specific amount of weight in a specific timeperiod (for example, lose 22 pounds or 10 kilograms in 3 months). Themedical conditions of the first user 118 may include one or morepre-existing diseases or conditions of the first user 118. Theexperience level of the first user 118 may indicate the number of hoursof the first fitness activity or an experience level associated with thefirst fitness activity.

In another embodiment, the received initial user input 304 may beassociated with the user profile of the first user 118. The user profileof the first user 118 may include the set of body parameters of thefirst user 118, the fitness goal of the first user 118, the medicalconditions of the first user 118, and the experience level of the firstuser 118, and other parameters that the first NN model 106 may requireto generate the first avatar 120 and the fitness routine of the firstuser 118. In some other embodiments, the circuitry 202 may be furtherconfigured to acquire a performance history of the first user 118associated with one or more previous fitness activities performed by thefirst user 118.

At 302B, a data acquisition operation may be performed. In the dataacquisition operation, the circuitry 202 may be configured to receive afirst user input 306 for selection of the first fitness activity from aset of fitness activities. The circuitry 202 may receive the selectionof the first fitness activity from suggestions of one or more fitnessactivities by the first NN model 106 based on the category of exercise(such as aerobics, Yoga, strength training, etc.) recommended in thefitness routine of the first user 118. For example, the circuitry 202may control the display device 108 to display a graphical user interface(such as a dropdown box or a selection box) that lists the one or morefitness activities for selection, and may receive the user selectionbased on user input via a remote control, a handheld controller, or atouch input. Based on the reception of the first user input 306, thecircuitry 202 may be configured to control the set of image sensors 104to capture the first set of images 116 of the first user 118. The firstset of images 116 may be captured for the duration in which the firstuser 118 may be engaged in the first fitness activity. In anotherembodiment, the circuitry 202 may be configured to control playback ofaudio content on one or more audio rendering devices associated with theelectronic device 102 based on the reception of the first user input306. The playback of the audio content may simulate a gymnasium ambiencefor the first fitness activity, and/or may set the tempo for the firstfitness activity.

At 302C, an augmented-reality display may be generated. In anembodiment, the circuitry 202 may be configured to generate theaugmented-reality display 114. The circuitry 202 may be configured togenerate the augmented-reality display 114 based on the first set ofimages 116 of the first user 118. The generated augmented-realitydisplay 114 may include the first avatar 120 and the live image of thefirst user 118 from the first set of images 116. In the generatedaugmented-reality display 114, the first avatar 120 and the live imageof the first user 118 may be combined in a real 3D world environment,where the first avatar 120 may enable real-time interaction with thefirst user 118.

In an embodiment, the first avatar 120 may be a virtual and animatedthree-dimensional (3D) graphical representation of the first user 118that may be customized to a persona of the first user 118. In anotherembodiment, the first avatar 120 may be a virtual and animatedthree-dimensional (3D) graphical representation of a generic human bodywith universal features. For example, the circuitry 202 may beconfigured to generate the first avatar 120 based on the first set ofimages 116, the set of body parameters of the first user 118, and theuser profile of the first user 118. The first avatar 120 may be apersonalized graphical representation of the first user 118, and may berepresented in either three-dimensional (3D) form or two-dimensional(2D) form. For example, the appearance of the first avatar 120 may besimilar to appearance of the first user 118. For example, a height and ashape of the body of the first avatar 120 may be similar to the heightand the shape of the body of the first user 118.

At 302D, an augmented-reality display rendering operation may beperformed. In the augmented-reality display rendering operation, thecircuitry 202 may be configured to control the display device 108 torender the generated augmented-reality display 114. The renderedaugmented-reality display 114 may include the first avatar 120 that maybe configured to demonstrate the first fitness activity on the displaydevice 108. For example, the circuitry 202 may control the first avatar120 according to pre-recorded animations associated with the selectedfirst fitness activity.

In another embodiment, the circuitry 202 may be configured to determinethe first position of the first user 118 in the augmented-realitydisplay 114 based on the captured first set of images 116. The circuitry202 may be configured to control the display device 108 to render thefirst avatar 120 at a second position in the augmented-reality display114 based on the determined first position. For example, the circuitry202 may control the display device 108 to display the first avatar 120adjacent to and spaced apart from the live image of the first user 118in the augmented-reality display 114. The distance between the liveimage of the first user 118 and the first avatar 120 may depend on thetype of the fitness activity, the number of users in theaugmented-reality display 114, or the screen real estate of the displaydevice 108. In an embodiment, the circuitry 202 may control the firstavatar 120 to start the first fitness activity based on a user cue suchthat the first user 118 may copy the movements of the first avatar 120to start preforming the first fitness activity. In an embodiment, in acase where the display device 108 is part of a smart phone, theaugmented-reality display 114 may be mirrored/casted on a larger displaydevice (such as a television) such for a better user experience.

In another embodiment, the circuitry 202 may be configured to receivesensor data associated with biological information of the body of thefirst user 118 during the performance of the first fitness activity bythe first user 118. In an embodiment, the sensor data may be receivedfrom a wearable device (such as a fitness tracker) or one or moresensors that may be worn by the first user 118 while the first user 118is engaged in the first fitness activity. Each of the one or moresensors may include suitable logic, circuitry, and/or interfaces thatmay be configured capture the sensor data associated with biologicalinformation (or biomarkers) and/or biomechanics of the body of the firstuser 118. Examples of such sensors may include, but are not limited to,a breathing rate sensor, a heart rate sensor, a pulse rate sensor, ablood pressure sensor, an oxygen saturation sensor, etc.

At 302E, a second NN model may be applied. In an embodiment, thecircuitry 202 may be configured to apply the second NN model 210 on thefirst set of images 116. The second NN model 210 may be applied on thefirst set of images 116 to determine posture information of the firstuser 118 for the duration of the first fitness activity. The second NNmodel 210 may output the posture information (or pose) of the first user118 for the duration of the first fitness activity. By way of exampleand not limitation, the second NN model 210 may correspond to OpenPosemodel or any other real-time pose estimation algorithm.

At 302F, posture information may be determined. In an embodiment, thecircuitry 202 may be configured to determine the posture information ofthe first user 118 for the duration of the first fitness activity, basedon the application of the second NN model 210. The posture informationmay indicate a posture of the first user 118 in each of the first set ofimages 116. For example, the posture information may include a pluralityof key points 308 corresponding to parts and/or joints (such as elbow,wrist, shoulder, neck, head, eyes, hip, knee, ankle, etc.) of the bodyof the first user 118.

FIG. 3B illustrates exemplary operations for personalized fitnessactivity training using augmented-reality based avatar, in accordancewith an embodiment of the disclosure. FIG. 3B is explained inconjunction with elements from FIGS. 1, 2, and 3A. With reference toFIG. 3B, there is shown a block diagram 300B that illustrates exemplaryoperations from 302G to 302J, as described herein. The exemplaryoperations illustrated in the block diagram 300B may start at 302G andmay be performed by any computing system, apparatus, or device, such asby the electronic device 102 of FIG. 1 or the circuitry 202 of FIG. 2 .Although illustrated with discrete blocks, the exemplary operationsassociated with one or more blocks of the block diagram 300B may bedivided into additional blocks, combined into fewer blocks, oreliminated, depending on the particular implementation.

At 302G, a set of key points may be determined. In an embodiment, thecircuitry 202 may be configured to determine a set of key points fromthe plurality of key points 308 based on the selected first fitnessactivity at 302B. The set of key points may be associated with aparticular fitness activity, and may differ from one fitness activity toanother. For example, a first set of key points 310 may be associatedwith the first fitness activity. For example, the first set of keypoints 310 may include key points related to elbows, wrists (or theradiocarpal joints), shoulder (or the glenohumeral joint), and the hipamong the plurality of key points 308 in a case where the first fitnessactivity is bicep curls. In another example, the first set of key points310 may include key points related to neck, shoulder, hip, and knees ina case where the first fitness activity is running or walking. In anembodiment, the circuitry 202 may determine the perspective or the angleof view from which the posture information and the first set of keypoints 310 are obtained. The circuitry 202 may determine the perspectiveto inform the first user 118 to change the position and orientation ofthe body of the first user 118 with respect to the set of image sensors104 such that the detected posture information is suitable for postureevaluation (for example, comparison with a reference posture). Inanother embodiment, the circuitry 202 may determine the perspective toapply a normalization process or a conversion process on the first setof key points 310 such that the detected posture information is suitablefor the posture evaluation.

At 302H, one or more angles between lines joining the set of key pointsmay be determined. In an embodiment, the circuitry 202 may be configuredto determine a set of coordinate values associated with the determinedfirst set of key points 310. Each of the set of coordinate value may beassociated with a corresponding key point of the first set of key points310 and may indicate a position of the corresponding key point withrespect to other key points in a 3D space. Based on the determined theset of coordinate values, the circuitry 202 may be configured todetermine one or more angles between lines 312 connecting the set ofcoordinate values. Typically, in bicep curls, a dumbbell is lifted up byan arm from a resting and extended position, with a rotation around theelbow, while other parts of the body are kept still. This action of thebicep curls targets the biceps muscle. One common mistake in bicep curlsincludes using the shoulder to help swing the weight up, thus causing arotation of the shoulder. Other common mistakes in bicep curls includelifting the weight partially or swinging the upper torso along with thearms. Accordingly, in a case where the first fitness activity is bicepcurls, the circuitry 202 may be configured to determine the anglebetween the upper arm (for example, the line joining the shoulder andelbow key points) and the torso (for example, the line joining theshoulder and hip key points) of the first user 118. The angle betweenthe upper arm and the torso may indicate whether the shoulder is rotatedwhile lifting the weight. The circuitry 202 may further determine theminimum angle between the upper arm and the forearm (for example, theline joining the wrist and elbow key points) of the first user 118. Theminimum angle between the upper arm and the forearm when the weight islifted up may indicate whether the extent of the lift is partial orfull. In an embodiment, the circuitry 202 may be configured to store thedetermined one or more angles in the memory 204 for further processing.

At 302I, the angles may be compared with reference angles. In anembodiment, the circuitry 202 may be configured to compare thedetermined one or more angles with a set of reference angles of areference posture 314. The reference posture may correspond to a postureof an experienced user who may be an expert in performing the firstfitness activity. In an embodiment, the reference posture 314 maycorrespond to a ground truth posture with respect to the first NN model106. The reference posture 314 may include the set of reference anglesbetween the first set of key points 310. By way of example, thereference posture may include a first reference angle and a secondreference angle. The first reference angle between the torso and upperarm may be in the range of 0 to 35 degrees, and may be characterized asa good posture for bicep curls by the first NN model 106. The secondreference angle between the forearm and upper arm when the weight islifted up may be in the range of 45 to 70 degrees, and may becharacterized as a good posture for bicep curls by the first NN model106. The comparison between the determined one or more angles and theset of reference angles may indicate a deviation of the posture of thefirst user 118 and the reference posture 314.

At 302J, a first NN model 106 may be applied. In an embodiment, thecircuitry 202 may be configured to apply the first NN model 106 on thedetermined first set of key points 310. The first NN model 106 may beconfigured to generate a classification result. In some embodiments, thefirst NN model 106 may generate the classification result based on thecomparison of the determined one or more angles associated with thefirst user 118 with the set of reference angles associated with thereference posture 314.

In another embodiment, the circuitry 202 may be configured to apply thefirst NN model 106 on the result of the comparison between thedetermined one or more angles associated with the first user 118 withthe set of reference angles. The first NN model 106 may output theclassification result based on the comparison result. The classificationresult may be used to classify or label the posture information of thefirst user 118 into one of a good posture or a bad posture based on thedetermined one or more angle between the first set of key points 310 andthe set of reference angles between the first set of key points 310. Inan embodiment, the algorithm associated with the first NN model 106 maybe modified during the training phase (or re-training phase) to adjust adecision boundary between good posture and bad posture to ease thestrictness for evaluation of the posture information. In anotherembodiment, the first NN model 106 may be trained with differenttraining datasets such that the decision boundary between good postureand bad posture may be changed at run time.

In another embodiment, the circuitry 202 may be configured to providethe first user profile of the first user 118 as input to the first NNmodel 106. For example, the first user profile may include informationabout the set of body parameters of the first user 118, the fitness goalof the first user 118, the medical condition of the first user 118, theexperience level of the first user 118 in performing the first fitnessactivity, the performance history of the first user 118 associated withone or more previous fitness activities of the first user 118, and thelike. In another embodiment, the circuitry 202 may be further configuredto apply the first NN model 106 on the user profile and the sensor dataassociated with the biological information of the first user 118.

Based on the application of the first NN model 106, the circuitry 202may classify the posture of the first user 118 as bad posture in a casewhere the determined angle between the upper arm and the forearm of thefirst user 118 is above 70 degrees, indicating that the weight is notlifted fully up. The circuitry 202 may classify the posture of the firstuser 118 as good posture in a case where the determined angle betweenthe upper arm and the forearm of the first user 118 is less than 70degrees. The circuitry 202 may further classify the posture of the firstuser 118 as bad posture in a case where the determined angle between theshoulder and upper arm is greater than 35 degrees, indicating that theshoulder is rotated to help carry the weight up. The circuitry 202 mayfurther classify the posture of the first user 118 as good posture in acase where the determined angle between the shoulder and upper arm isless than 35 degrees.

In some embodiments, the first NN model 106 may be trained to classifythe posture information associated with the first fitness activity. Insuch scenarios, the electronic device 102 may include a set of neuralnetwork models each corresponding to a fitness activity of a pluralityof fitness activities. In another embodiment, the first NN model 106 maybe trained to classify the posture information associated with the eachof the plurality of fitness activities.

FIG. 3C illustrates exemplary operations and user interface forpersonalized fitness activity training using augmented-reality basedavatar, in accordance with an embodiment of the disclosure. FIG. 3C isexplained in conjunction with elements from FIGS. 1, 2, 3A, and 3B. Withreference to FIG. 3C, there is shown a block diagram 300C thatillustrates exemplary operations from 302K to 302M, as described herein.The exemplary operations illustrated in the block diagram 300C may startat 302K and may be performed by any computing system, apparatus, ordevice, such as by the electronic device 102 of FIG. 1 or the circuitry202 of FIG. 2 . Although illustrated with discrete blocks, the exemplaryoperations associated with one or more blocks of the block diagram 300Cmay be divided into additional blocks, combined into fewer blocks, oreliminated, depending on the particular implementation.

At 302K, a real-time feedback may be determined. In an embodiment, thecircuitry 202 may be configured to determine the real-time feedbackbased on the application of the first NN model 106 on the postureinformation and the determined first set of key points 310. Thereal-time feedback may be determined in response to performance of thefirst fitness activity by the first user 118. For example, circuitry 202may determine the real-time feedback based on the classification resultassociated with the posture of the first user 118. In an embodiment, thedetermined real-time feedback may be associated with at least one of amovement of one or more parts of a body of the first user 118, theposture of the first user 118, a number of repetitions of the firstfitness activity, the duration of the first fitness activity, or abreathing pattern of the first user 118 during the first fitnessactivity. In an example, the real-time feedback associated with themovement of one or more parts of the body of the first user 118 mayinform the first user 118 to lift the weight higher such that the anglebetween the upper arm and the forearm is less than 70 degrees. Inanother example, the real-time feedback associated with the posture ofthe first user 118 may inform the first user 118 to limit the rotationof the shoulder during bicep curls such that the bicep curls effectivelytargets the biceps muscle. In another example, the real-time feedbackassociated with the posture of the first user 118 may inform the firstuser 118 to keep user's back straight during the first fitness activitysuch that the first user 118 does not injure user's back.

In another example, the real-time feedback associated with the number ofrepetitions of the first fitness activity and the duration of the firstfitness activity may inform the first user 118 to increase or decreasethe number of repetitions or the duration of the first fitness activity.For example, in the first attempt, the first user 118 may perform thefirst fitness activity for a first duration (for example, 30 seconds) ormay perform the first fitness activity for a first number of repetitions(for example, 10 repetitions). In the second attempt, the circuitry 202may determine a second duration (for example, 60 seconds) or a secondnumber of repetitions (for example, 12 repetitions) of the first fitnessactivity as the real-time feedback. In an embodiment, the real-timefeedback associated with the number of repetitions of the first fitnessactivity and the duration of the first fitness activity may be based onthe fitness goal of the first user 118, the medical condition of thefirst user 118, or the experience level of the first user 118 inperforming the first fitness activity.

In another example, the real-time feedback associated with a breathingpattern of the first user 118 may inform the first user 118 to changeuser's breathing pattern (breathing cycle) from a first breathingpattern to a second breathing pattern. For example, the circuitry 202may detect that the first user 118 may be exhaling while lifting theweight and inhaling while dropping the weight in the first breathingpattern. The circuitry 202 may determine the second breathing pattern,in which the first user 118 has to inhale while lifting the weight andexhale while dropping the weight, as the real-time feedback. In anotherexample, the real-time feedback associated with the breathing pattern ofthe first user 118 may inform the first user 118 to consciously continuebreathing while performing the first fitness activity, based on thedetected breathing pattern of the first user 118.

In another embodiment, the determined real-time feedback may include amotivational phrase based on the performance of the first user 118. Forexample, in a case where the movement of one or more parts of the bodyof the first user 118, the posture of the first user 118, the number ofrepetitions of the first fitness activity, the duration of the firstfitness activity, and the breathing pattern of the first user during thefirst fitness activity is close to ideal reference values, thedetermined real-time feedback may include the motivational phrase toencourage the first user 118 to continue the same exercise pattern. Inanother example, the determined real-time feedback may include themotivational phrase to motivate the first user 118 to complete therequired number of repetitions.

At 302L, the determined real-time feedback may be rendered. In anembodiment, the circuitry 202 may be configured to control the firstavatar 120 to output the determined real-time feedback in theaugmented-reality display 114. In an embodiment, the circuitry 202 maybe configured to output the determined real-time feedback in the form ofone of the movement of the first avatar 120 in the augmented-realitydisplay 114, a synthesized speech, or a textual feedback. In a casewhere the circuitry 202 determines a deviation between the posture ofthe user and the reference posture 314, the circuitry 202 may controlthe first avatar 120 to demonstrate the correct posture such that thefirst user 118 may copy the movement of the first avatar 120. Theposture and movement of the first avatar 120 may be similar to theposture and movement of an expert in performing the first fitnessactivity. In an embodiment, the circuitry 202 may control the displaydevice 108 to pause the ongoing animation of the first avatar 120, andcause the first avatar 120 to re-demonstrate the correct posture for oneor more portions of the first fitness activity using other animations(such as other views or closeup views). The electronic device 102 mayalso output verbal feedback (such as feedback on posture, feedback ontempo, feedback on breathing pattern, and so on).

In a case where the real-time feedback is associated with the number ofrepetitions of the first fitness activity, the duration of the firstfitness activity, or the breathing pattern of the first user 118 duringthe first fitness activity, the circuitry 202 may be configured tooutput the determined real-time feedback as the synthesized speechand/or the textual feedback. The circuitry 202 may be configured tooutput the motivational phrase as the synthesized speech. In someembodiments, the circuitry 202 may control an audio rendering device(such as speakers) associated with the electronic device 102 to outputthe motivational phrase.

In an embodiment, the circuitry 202 may generate and output a graphicaluser interface 316 on the display device 108 to render the determinedreal-time feedback. The graphical user interface 316 may include theaugmented-reality display 114, the textual feedback 316A, and one ormore UI elements. For example, the textual feedback 316A may include thephrase “Exercise performed correctly”, and may indicate that the posturerelated to a specific body part has been corrected. The textual feedback316A may further indicate whether the torso is straight, the anglebetween the torso and the upper arm, and so on. For example, the one ormore UI elements of the graphical user interface 316 may include acountdown timer for counting down the duration of the first fitnessactivity, a progress bar showing the progress of the first user 118 withrespect to the first fitness activity and the transitions betweenfitness activities, an icon or text indicating the type of the firstfitness activity along with the number of repetitions and the weight ofthe dumbbell. The one or more UI elements of the graphical userinterface 316 may further include a statistics bar 3168 that indicatesthe number of repetitions performed, the pace at which the first fitnessactivity is performed, the depth of the first fitness activity, heartrate of the first user 118, and so on. The one or more UI elements ofthe graphical user interface 316 may further include an icon 316C “talkto me” that may be pressed to receive speech input from the first user118 to ask the first avatar 120 to demonstrate the first fitnessactivity or verbally guide the first user 118 through the first fitnessactivity. The graphical user interface 316 may further include aleaderboard that may list the top performers of the first fitnessactivity in terms of number of repetitions or number of hours over aperiod of time.

At 302M, a second fitness activity may be selected. In an embodiment,the circuitry 202 may receive a second user input for selection of thesecond fitness activity (for example, reverse curl or dumbbell chestpress) from the set of fitness activities. The circuitry 202 may befurther configured to determine a second set of key points from theplurality of key points 308 based on the selection of the second fitnessactivity. The second set of key points may be different than the firstset of key points 310. The circuitry 202 may be further configured toapply the first NN model 106 on the second set of key points to classifythe posture of the first user 118. The circuitry 202 may be configuredto determine first real-time feedback based on the classification, andmay control the first avatar 120 to output the determined firstreal-time feedback in the augmented-reality display 114.

In another embodiment, the circuitry 202 may detect a set of users (forexample two or more users) engaged in the first fitness activity. Insuch scenario, the circuitry 202 may be configured to receive a secondset of images of the set of users engaged in the first fitness activity.The set of users may include the first user 118 and a second user. Thecircuitry 202 may be further configured to generate theaugmented-reality display 114 that may include a set of avatarsassociated with the set of users. The set of avatars may include thefirst avatar 120 and a second avatar. The first avatar 120 may beassociated with the first user 118, and may be configured to interactwith the first user 118 in the augmented-reality display 114. The secondavatar may be associated with the second user, and may be configured tointeract with the second user in the augmented-reality display 114. Thecircuitry 202 may be further configured to determine the postureinformation associated with each user of the set of users based on thesecond set of images and the first fitness activity. The circuitry 202may be further configured to determine the real-time feedback for eachuser of the set of users based on application of the first NN model 106on the determined posture information associated with each user of theset of users. The circuitry 202 may be further configured to control thefirst avatar 120 and the second avatar to output the determinedreal-time feedback in the augmented-reality display 114. In anotherembodiment, the circuitry 202 may be configured to generate theaugmented-reality display 114 that may include a single avatar (forexample, the first avatar 120) with universal features, and may controlthe single avatar to guide and interact with each user of the set ofusers.

FIG. 4 is a diagram that illustrates exemplary operations for generationof a fitness routine and activity schedule for a first user, inaccordance with an embodiment of the disclosure. FIG. 4 is explained inconjunction with elements from FIG. 1 , FIGS. 2, and 3A-3C. Withreference to FIG. 4 , there is shown a block diagram 400 thatillustrates exemplary operations from 402A to 402F, as described herein.The exemplary operations illustrated in the block diagram 400 may startat 402A and may be performed by any computing system, apparatus, ordevice, such as by the electronic device 102 of FIG. 1 or the circuitry202 of FIG. 2 . Although illustrated with discrete blocks, the exemplaryoperations associated with one or more blocks of the block diagram 400may be divided into additional blocks, combined into fewer blocks, oreliminated, depending on the particular implementation.

At 402A, data acquisition may be performed. In an embodiment, thecircuitry 202 may be configured to receive an initial user input 404associated with at least one of the set of body parameters of the firstuser 118, the user profile of the first user 118, the fitness goal ofthe first user 118, the medical condition of the first user 118, or theexperience level of the first user 118 in performing the first fitnessactivity. The set of body parameters of the first user 118 may includean age of the first user 118, a height of the first user 118, a weightof the first user 118, a gender of the first user 118, a waist size ofthe first user 118, a size of arms of the first user 118, and the like.The fitness goal of the first user 118 may be correspond to a goal ofthe first user 118 associated with the fitness of the first user 118 andmay be associated with a time period. The medical conditions of thefirst user 118 may include one or more pre-existing diseases of thefirst user 118. The experience level of the first user 118 may indicatean experience of the first user 118 in performing the first fitnessactivity. In an embodiment, the user profile of the first user 118 mayinclude information related to the set of body parameters of the firstuser 118, the fitness goal of the first user 118, the medical conditionsof the first user 118, the experience level of the first user 118, andso on.

At 402B, a performance history may be acquired. In an embodiment, thecircuitry 202 may be configured to acquire the performance history ofthe first user 118. The performance history of the first user 118 may beassociated with one or more previous fitness activities of the firstuser 118. In an embodiment, the one or more previous fitness activitiesmay include the first fitness activity. The performance history of theone or more previous fitness activities may include a name of thefitness activity, a number of days for which the fitness activity hasbeen performed by the first user 118, a time duration for which thecorresponding fitness activity has been performed per session, a numberof repetitions of the fitness activity, one or more injuries sustainedby the first user 118 due to the fitness activity, the past fitness goalassociated with fitness activity, and the like.

At 402C, a fitness routine may be generated. In an embodiment, thecircuitry 202 may be configured to generate the fitness routine for thefirst user 118 based on one of the received initial user input 404 orthe acquired performance history. The generated fitness routine mayinclude a suggestion of one or more potential fitness activities basedon at least one of the received user input and/or the performancehistory. The one or more potential fitness activities may include thefirst fitness activity. In an embodiment, the one or more potentialfitness activities may be suggested to achieve the fitness goal. Inanother embodiment, the circuitry 202 may generate a diet plan for thefirst user 118 that augments with the fitness routine. The diet plan mayinclude information related a variety of foods that have to be consumedby the first user 118 to achieve the fitness goal. The diet plan mayalso include time of day information associated with the food to beconsumed by the first user 118. Examples of the fitness routine and thediet plan are shown in Table 1.

At 402D, the fitness routine may be rendered. In an embodiment, thecircuitry 202 may be configured to output the generated fitness routineon the display device 108 or a user device (for example, a mobile phone)associated with the first user 118. In an embodiment, the circuitry 202may be configured to output a detailed form of the fitness routine onthe graphical user interface, shown in FIG. 3C.

At 402E, an activity schedule may be determined. In an embodiment, thecircuitry 202 may be configured to determine the activity schedule forperforming the first fitness activity by the first user 118. Theactivity schedule may be determined based on the generated fitnessroutine. In an embodiment, the determined activity schedule may includea number of days/months for which the first fitness activity is to beperformed, time information that may indicate a schedule (for example,time of day, duration, etc.) for the first fitness activity, andrepetition information that may indicate a number of repetition of thefirst fitness activity. Examples of the activity schedule according tothe fitness goal are shown in Table 1.

TABLE 1 Examples of Fitness Routine and Activity Schedule Determined bythe Application of the First NN model 106 According to Fitness GoalFitness Goal Fitness routine Activity schedule Diet plan Lose 22 Aerobicexercises/ Mon 8 AM: Warm up + 3 meals a day pounds/10 Cardio/StrengthCardio (dance form) for spaced at least kilograms training 4 days a 15mins 5 hours apart in 3 week. Tue 8 AM: Warm up + with one meal monthsYoga 2 days a Spin class for 15 mins including salad week Wed 8 AM: Yogaonly. Reduce Thu 8 AM: Warm up + calorie intake Treadmill for 15 mins by20% in a Fri 8 AM: Yoga week. Sat: Warm up + Strength Exercise for 15mins Sun: Rest Gain 10 Weight training/ Mon 5 PM: Warm up + 4 meals aday pounds/5 Strength training Biceps and triceps spaced at leastkilograms 5 days a week. workout 30 reps each 4 hours apart in muscleTue 5 PM: Warm up + including one mass in 2 Chest workout 30 reps mealrich in months Wed 5 PM: Warm up + protein (meat, Abs workout 30 repseggs, almonds, Thu: Rest sprouts, etc.) Fri 5 PM: Warm up + and onesnack Legs workout 40 reps one hour before Sat 5 PM: Warm up + exercise.Shoulder workout 40 reps Increase calorie Sun: Rest intake by 20% in aweek. Optional protein supplements twice a week with breakfast. Run 5kRunning/ Mon 7 AM: Warm up + 3 meals a day marathon Technique/ treadmillfor 30 mins spaced at least in 6 Strength training (includes posturetraining) 5 hours apart months 5 days a week. Tue 7 AM: Warm up + withone meal 600m run rich in protein Wed 7 AM: Rest (meat, eggs, Thu 7 AM:Indoor almonds, strength training for legs sprouts, etc.). Fri 7 AM:Warm up + Increase calorie 800m run intake by 20% Sat 7 AM: Warm up + ina week. 1K run Sun: Rest

As shown in table 1, the circuitry 202 may receive the fitness goal asuser input, and may determine the fitness routine and correspondingactivity schedule and diet plan for achieving the fitness goal. Forexample, in a case where the fitness goal includes “Lose 22 pounds/10kilograms in 3 months”, the circuitry 202 may apply the first NN model106 on the user input and the other factors (for example, the set ofbody parameters such as weight and height, the user profile, the medicalcondition, the experience level, age, gender, and so on) to determinethe optimum activity schedule and diet plan for achieving the fitnessgoal. For example, in a case where the fitness goal includes losingweight, the circuitry 202 may determine a fitness routine that includesaerobics/strength training for 4 days a week and Yoga for 2 days a week.In a case where the fitness goal includes gaining weight through anincrease in muscle mass, the circuitry 202 may determine a fitnessroutine that includes weight training and strength training for 5 days aweek. In a case where the fitness goal includes running a marathon, thecircuitry 202 may determine a fitness routine that includes running,technique, and strength training for 5 days a week. Table 1 showsexamples of the corresponding activity schedule and diet plan that maybe determined by the application of the first NN model 106, and mayemulate the fitness routine and the activity schedule that may be set byan experienced trainer. It may be noted that the first NN model 106 maydetermine different fitness routines and diet plans for user withdifferent body parameters, ages, gender, and so on.

At 402F, one or more reminders may be generated. In an embodiment, thecircuitry 202 may be configured to periodically generate and output oneor more notifications to the user device associated with the first user118 based on the determined activity schedule. The notifications mayinclude one of a reminder to perform the first fitness activity or astatus of the first fitness activity with respect to the fitness goal.For example, the circuitry 202 may be configured to output a firstnotification (for example, “exercise in 10 minutes. Get ready!) toremind the first user 118 about the first fitness activity before thescheduled time according to the generated activity schedule. In anotherexample, in a case where the fitness goal is to gain muscle mass, thecircuitry 202 may be configured to output a second notification (forexample, “It's almost 4 PM. Have a healthy snack now!) to remind thefirst user 118 to consume a healthy snack one hour before the exerciseat 5 PM according to the diet plan.

FIG. 5 is a diagram that illustrates training of a first neural networkmodel to determine real-time feedback for a first fitness activity of afirst user, in accordance with an embodiment of the disclosure. FIG. 5is explained in conjunction with elements from FIGS. 1, 2, 3A-3C, and 4. With reference to FIG. 5 , there is shown a block diagram 500. In theblock diagram 500, there is shown a first neural network (NN) model 502,and a training dataset 504. The training dataset 504 may include aplurality of training samples 506. The plurality of training samplesincludes a first training sample 506A, a second training sample 506B, upto an Nth training sample 506N.

In an embodiment, the circuitry 202 may receive, from the server 110,the training dataset 504 that may include the plurality of trainingsamples 506 for training of the first NN model 502. In anotherembodiment, the circuitry 202 may generate the training dataset 504 thatmay include the plurality of training samples 506 for training of thefirst NN model 502. Each of the plurality of training samples 506 mayinclude one or more features that may be used by the first NN model 106to determine real-time feedback for the first user 118. The one or morefeatures may include, but is not limited to, a set of key points of theposture information for each fitness activity of a plurality of fitnessactivities, a movement of one or more body parts for each fitnessactivity, a number of repetitions for each fitness activity, a durationof each fitness activity, a breathing pattern associated with eachfitness activity, and so on.

In an embodiment, the first training sample 506A may include key pointinformation 508A associated with the set of key points of the idealposture information for each fitness activity of the plurality offitness activities, movement information 508B associated with the idealmovements of the one or more body parts for each fitness activity,repetition information 508C associated with the ideal number ofrepetitions for each fitness activity, duration information 508Dassociated with the ideal duration of each fitness activity, andbreathing information 508E associated with the ideal breathing patternassociated with each fitness activity.

In an embodiment, the first NN model 502 may be trained to determine thereal-time feedback for a single fitness activity from the plurality offitness activities. For example, the first NN model 502 may be trainedto determine the real-time feedback associated with the first fitnessactivity (for example, bicep curls). To determine the real-time feedbackassociated with the first fitness activity, the first NN model 502 maybe trained on the one or more features that may include the postureinformation for the first fitness activity, the movement of one or morebody parts for the first fitness activity, the number of repetitions forthe first fitness activity, the duration of the first fitness activity,the breathing pattern associated with the first fitness activity, and soon. In another embodiment, the first NN model 502 may be trained toclassify the posture information for the plurality of fitness activities(for example, bicep curls, reverse curls, preacher curls, and so on) fora particular body part. In another embodiment, the first NN model 502may be trained to classify the posture information for the plurality offitness activities (for example, strength training, aerobics, Yoga,dance forms, and so on).

In an embodiment, the electronic device 102 may include a plurality ofthe neural network models. Each of the plurality of neural networkmodels may be trained to classify the posture information of the userassociated with each of the plurality of fitness activities. In anotherembodiment, a single neural network model of the plurality of neuralnetwork models may be trained to classify the posture informationassociated with one or more fitness activities of the plurality ofactivities. For example, a third NN model may be trained to classify theposture information associated with a second fitness activity (forexample, dumbbell press). To classify the posture information associatedwith the second fitness activity, the third NN model may be trained onone or more features that may include the posture information for thesecond fitness activity, the movement of one or more body parts for thesecond fitness activity, the number of repetitions for the secondfitness activity, the duration of the second fitness activity, and thebreathing pattern associated with the second fitness activity.

FIG. 6 is a first flowchart that illustrates exemplary operations forpersonalized fitness activity training using augmented-reality basedavatar, in accordance with an embodiment of the disclosure. FIG. 6 isexplained in conjunction with elements from FIGS. 1, 2, 3A-3C, 4, and 5. With reference to FIG. 6 , there is shown a flowchart 600. Theoperations from 602 to 614 may be implemented on any computing device,for example, the electronic device 102 or the circuitry 202. Theoperations may start at 602 and may proceed to 604.

At 604, a first set of images 116 of the first user 118 may be received,wherein the first set of images 116 may be captured for the duration inwhich the first user 118 may be engaged in the first fitness activity.In one or more embodiments, the circuitry 202 may be configured toreceive the first set of images 116 of the first user 118. The detailsof the reception of the first set of images 116 of the first user 118are provided for example, in FIGS. 1, and 3A.

At 606, an augmented-reality display 114 that includes the first avatar120 and the image (for example, a live image) of the first user 118 maybe generated based on the first set of images 116. In one or moreembodiments, the circuitry 202 may be configured to generate theaugmented-reality display 114 that includes the first avatar 120 and theimage of the first user 118 based on the first set of images 116. Thedetails of the generation of the augmented-reality display 114 areprovided, for example, in FIGS. 1, and 3A.

At 608, a display device 108 may be controlled to render the generatedaugmented-reality display 114, wherein the rendered augmented-realitydisplay 114 may include the first avatar 120 configured to perform thefirst fitness activity. In one or more embodiments, the circuitry 202may be configured to control the display device 108 to render thegenerated augmented-reality display 114. The details of controlling thedisplay device 108 are provided, for example, in FIGS. 1 and 3C.

At 610, posture information of the first user 118 may be determinedbased on the first set of images 116 for the duration in which the firstuser 118 may be engaged in the first fitness activity. In one or moreembodiments, the circuitry 202 may be configured to determine theposture information of the first user 118 based on the first set ofimages 116 for the duration in which the first user 118 may be engagedin the first fitness activity. The details of the determination of theposture information are provided, for example, in FIG. 3 .

At 612, real-time feedback may be determined based on application of afirst neural network (NN) model 106 on the determined postureinformation, wherein the determination of the real-time feedback may bein response to performance of the first fitness activity by the firstuser 118. In one or more embodiments, the circuitry 202 may beconfigured to determine the real-time feedback based on application ofthe first NN model 106 on the determined posture information, asdescribed in FIGS. 1, 3A-3C, and 5 .

At 614, the first avatar 120 may be controlled to output the determinedreal-time feedback in the augmented-reality display 114. In one or moreembodiments, the circuitry 202 may be configured to control the firstavatar 120 to output the determined real-time feedback in theaugmented-reality display 114. The details of the control of the firstavatar 120 to output the determined real-time feedback in theaugmented-reality display 114 are provided, for example, in FIGS. 1 and3C. Control may pass to end.

Various embodiments of the disclosure may provide a non-transitorycomputer-readable medium and/or storage medium having stored thereon,instructions executable by a machine and/or a computer such as theelectronic device 102. The instructions may cause the machine and/orcomputer to perform operations that may include reception of the firstset of images 116 of the first user 118. The first set of images 116 maybe captured for the duration in which the first user 118 is engaged inthe first fitness activity. The operations may further includegeneration of the augmented-reality display 114 that may include thefirst avatar 120 and the image of the first user 118 based on the firstset of images 116. The operations may further include controlling thedisplay device 108 to render the generated augmented-reality display114. The rendered augmented-reality display 114 may include the firstavatar 120 configured to perform the first fitness activity. Theoperations may further include determining the posture information ofthe first user 118 based on the first set of images 116 for the durationin which the first user 118 may be engaged in the first fitnessactivity. The operations may further include determining real-timefeedback based on application of the first neural network (NN) model 106on the determined posture information. The determination of thereal-time feedback may be in response to performance of the firstfitness activity by the first user 118. The operations may furtherinclude controlling the first avatar 120 to output the determinedreal-time feedback in the augmented-reality display 114.

Exemplary aspects of the disclosure may include an electronic device(such as the electronic device 102 of FIG. 1 ) that may includecircuitry (such as the circuitry 202). The circuitry 202 may beconfigured to control an image sensor of a set of image sensors (such asthe set of image sensors 104) to capture a first set of images (such asthe first set of images 116). The circuitry 202 may further receive thefirst set of images 116 of a first user (such as the first user 118).The first set of images 116 may be captured for a duration in which thefirst user 118 may be engaged in a first fitness activity. The circuitry202 may be configured to generate an augmented-reality display (such asthe augmented-reality display 114) that may include a first avatar (suchas the first avatar 120) and an image of the first user 118 based on thefirst set of images 116. The circuitry 202 may be configured todetermine a position of the first user 118 in the augmented-realitydisplay 114 based on the captured first set of images 116. The circuitry202 may be configured to control a display device (such as the displaydevice 108) to render the generated augmented-reality display 114. Therendered augmented-reality display 114 may include the first avatar 120that may be configured to perform the first fitness activity. In anembodiment, the circuitry 202 may be configured to control the displaydevice 108 to render the first avatar 120 in the augmented-realitydisplay 114 based on the determined position. In an embodiment, thecircuitry 202 may be configured to detect a specific gesture of thefirst user based on the first set of images 116 to trigger the renderedfirst avatar 120 to start the first fitness activity.

In accordance with an embodiment, the circuitry 202 may be configured toapply a second neural network model (such as the second NN model 210) onthe first set of images 116 to determine the posture information of thefirst user 118 for the duration of the first fitness activity. Theposture information may include a plurality of key points (such as theplurality of key points 308) corresponding to joints of a body of thefirst user 118. The circuitry 202 may be configured to receive a userinput for selection of the first fitness activity from a set of fitnessactivities. The circuitry 202 may be further configured to determine afirst set of key points 310 from the plurality of key points 308 basedon the selection of the first fitness activity. The circuitry 202 may befurther configured to apply a first neural network model (such as thefirst NN model 106) on the determined first set of key points 310 toclassify the posture information of the first user 118. The circuitry202 may be further configured to determine the real-time feedback basedon the classification.

In accordance with an embodiment, the circuitry 202 may be configured todetermine a set of coordinate values associated with the determinedfirst set of key points 310. The circuitry 202 may be further configuredto determine one or more angles between lines connecting the set ofcoordinate values. The circuitry 202 may be further configured tocompare the determined one or more angles with a reference posture,wherein the reference posture includes a set of reference angles betweenthe first set of key points 310. The circuitry 202 may further generatea classification result based on the comparison. The circuitry 202 maybe further configured to determine the real-time feedback based on theclassification result.

In accordance with an embodiment, the circuitry 202 may be configured toreceive a user input for selection of a second fitness activity from aset of fitness activities. The circuitry 202 may be configured todetermine a second set of key points from the plurality of key pointsbased on the selection of the second fitness activity. The circuitry 202may be further configured to apply the first NN model 106 on the secondset of key points to classify the posture information of the first user118. The circuitry 202 may be further configured to determine thereal-time feedback based on the classification.

In accordance with an embodiment, the circuitry 202 may be furtherconfigured to receive sensor data associated with biological informationof a body of the first user 118. The circuitry 202 may be furtherconfigured to determine the real-time feedback based on the receivedsensor data. In an embodiment, the determination of the real-timefeedback may be in response to performance of the first fitness activityby the first user 118.

In accordance with an embodiment, the circuitry 202 may be configured tocontrol the first avatar 120 to output the determined real-time feedbackin the augmented-reality display 114. In another embodiment, thecircuitry 202 may be further configured to output the determinedreal-time feedback as one of a movement of the first avatar 120 in theaugmented-reality display 114, a synthesized speech, or a textualfeedback. The determined real-time feedback may be associated with atleast one of a movement of one or more parts of a body of the first user118, a posture of the first user 118, a number of repetitions of thefirst fitness activity, the duration of the first fitness activity, or abreathing pattern of the first user 118 during the first fitnessactivity. In another embodiment, the determined real-time feedbackfurther may include a motivational phrase based on the performance ofthe first user 118. The circuitry 202 may be further configured tooutput the motivational phrase as a synthesized speech or a textualfeedback.

In accordance with an embodiment, the circuitry 202 may be furtherconfigured to receive a user input associated with at least one of a setof body parameters of the first user 118, a user profile of the firstuser 118, a fitness goal of the first user 118, a medical condition ofthe first user 118, or an experience level of the first user 118 inperforming the first fitness activity. The circuitry 202 may be furtherconfigured to acquire a performance history of the first user 118associated with one or more previous fitness activities of the firstuser 118. The circuitry 202 may be further configured to generate afitness routine that may include a suggestion of one or more potentialfitness activities based on at least one of the received user input orthe performance history. The one or more potential fitness activitiesmay include the first fitness activity. The circuitry 202 may beconfigured to output the fitness routine on one of the display device108 or a user device associated with the first user 118. In anembodiment, the circuitry 202 may be further configured to generate thefirst avatar 120 based on the set of body parameters of the first user118 and the user profile of the first user 118.

In accordance with an embodiment, the circuitry 202 may be configured todetermine an activity schedule for performing the first fitness activityby the first user 118 based on the fitness routine. The circuitry 202may be configured to output one or more notifications periodically tothe user device based on the determined activity schedule. Thenotifications may include one of a reminder to perform the first fitnessactivity or a status of the first fitness activity with respect to thefitness goal.

In accordance with an embodiment, the circuitry 202 may be configured toreceive a second set of images of a set of users engaged in the firstfitness activity. The set of users may include the first user 118 and asecond user. The circuitry 202 may be further configured to generate theaugmented-reality display 114 that includes a set of avatars associatedwith the set of users. The set of avatars may include the first avatarand a second avatar. The circuitry 202 may be further configured todetermine the posture information associated with each user of the setof users based on the second set of images and the first fitnessactivity. The circuitry 202 may be further configured to generate thereal-time feedback for each user of the set of users based on thedetermined posture information.

In accordance with an embodiment, the circuitry 202 may be configured totrain the first NN model 106 on one or more features to classify theposture information of the first user 118 and to determine the real-timefeedback. The one or more features may include a set of key points ofthe posture information for each fitness activity of a plurality offitness activities, a movement of one or more body parts for eachfitness activity, a number of repetitions for each fitness activity, theduration of each fitness activity, and a breathing pattern associatedwith each fitness activity.

The present disclosure may be realized in hardware, or a combination ofhardware and software. The present disclosure may be realized in acentralized fashion, in at least one computer system, or in adistributed fashion, where different elements may be spread acrossseveral interconnected computer systems. A computer system or otherapparatus adapted to carry out the methods described herein may besuited. A combination of hardware and software may be a general-purposecomputer system with a computer program that, when loaded and executed,may control the computer system such that it carries out the methodsdescribed herein. The present disclosure may be realized in hardwarethat comprises a portion of an integrated circuit that also performsother functions.

The present disclosure may also be embedded in a computer programproduct, which comprises all the features that enable the implementationof the methods described herein, and which when loaded in a computersystem is able to carry out these methods. Computer program, in thepresent context, means any expression, in any language, code ornotation, of a set of instructions intended to cause a system withinformation processing capability to perform a particular functioneither directly, or after either or both of the following: a) conversionto another language, code or notation; b) reproduction in a differentmaterial form.

While the present disclosure is described with reference to certainembodiments, it will be understood by those skilled in the art thatvarious changes may be made, and equivalents may be substituted withoutdeparture from the scope of the present disclosure. In addition, manymodifications may be made to adapt a particular situation or material tothe teachings of the present disclosure without departure from itsscope. Therefore, it is intended that the present disclosure is notlimited to the particular embodiment disclosed, but that the presentdisclosure will include all embodiments that fall within the scope ofthe appended claims.

What is claimed is:
 1. An electronic device, comprising: circuitryconfigured to: receive a first set of images of a first user, whereinthe first set of images is captured for a duration in which the firstuser is engaged in a first fitness activity; generate anaugmented-reality display that includes a first avatar and an image ofthe first user based on the first set of images; control a displaydevice to render the generated augmented-reality display, wherein therendered augmented-reality display includes the first avatar configuredto perform the first fitness activity; determine posture information ofthe first user based on the first set of images for the duration inwhich the first user is engaged in the first fitness activity; determinereal-time feedback based on application of a first neural network modelon the determined posture information, wherein the determination of thereal-time feedback is in response to performance of the first fitnessactivity by the first user; and control the first avatar to output thedetermined real-time feedback in the augmented-reality display.
 2. Theelectronic device according to claim 1, wherein the circuitry is furtherconfigured to apply a second neural network on the first set of imagesto determine the posture information of the first user for the durationof the first fitness activity, and the posture information includes aplurality of key points corresponding to joints of a body of the firstuser.
 3. The electronic device according to claim 2, wherein thecircuitry is further configured to: receive a user input for selectionof the first fitness activity from a set of fitness activities;determine a first set of key points from the plurality of key pointsbased on the selection of the first fitness activity; apply the firstneural network model on the determined first set of key points toclassify the posture information of the first user; and determine thereal-time feedback based on the classification.
 4. The electronic deviceaccording to claim 3, wherein the circuitry is further configured to:determine a set of coordinate values associated with the determinedfirst set of key points; determine one or more angles between linesconnecting the set of coordinate values; compare the determined one ormore angles with a reference posture, wherein the reference postureincludes a set of reference angles between the first set of key points;generate a classification result based on the comparison; and determinethe real-time feedback based on the classification result.
 5. Theelectronic device according to claim 2, wherein the circuitry is furtherconfigured to: receive a user input for selection of a second fitnessactivity from a set of fitness activities; determine a second set of keypoints from the plurality of key points based on the selection of thesecond fitness activity; apply the first neural network model on thesecond set of key points to classify the posture information of thefirst user; and determine the real-time feedback based on theclassification.
 6. The electronic device according to claim 1, whereinthe circuitry is further configured to: control an image sensor tocapture the first set of images; determine a position of the first userin the augmented-reality display based on the captured first set ofimages; control the display device to render the first avatar in theaugmented-reality display based on the determined position; and detect aspecific gesture of the first user based on the first set of images totrigger the rendered first avatar to start the first fitness activity.7. The electronic device according to claim 1, wherein the circuitry isfurther configured to output the determined real-time feedback as one ofa movement of the first avatar in the augmented-reality display, asynthesized speech, or a textual feedback.
 8. The electronic deviceaccording to claim 1, wherein the determined real-time feedback isassociated with at least one of a movement of one or more parts of abody of the first user, a posture of the first user, a number ofrepetitions of the first fitness activity, the duration of the firstfitness activity, or a breathing pattern of the first user during thefirst fitness activity.
 9. The electronic device according to claim 8,wherein the determined real-time feedback further comprises amotivational phrase based on the performance of the first user; and thecircuitry is further configured to output the motivational phrase as asynthesized speech or a textual feedback.
 10. The electronic deviceaccording to claim 1, wherein the circuitry is further configured to:receive a user input associated with at least one of a set of bodyparameters of the first user, a user profile of the first user, afitness goal of the first user, a medical condition of the first user,or an experience level of the first user in performing the first fitnessactivity; acquire a performance history of the first user associatedwith one or more previous fitness activities of the first user; generatea fitness routine that includes a suggestion of one or more potentialfitness activities based on at least one of the received user input orthe performance history, wherein the one or more potential fitnessactivities includes the first fitness activity; and output the fitnessroutine on one of the display device or a user device associated withthe first user.
 11. The electronic device according to claim 10, whereinthe circuitry is further configured to: determine, based on the fitnessroutine, an activity schedule for performing the first fitness activityby the first user; and output one or more notifications periodically tothe user device based on the determined activity schedule, wherein thenotifications include one of a reminder to perform the first fitnessactivity or a status of the first fitness activity with respect to thefitness goal.
 12. The electronic device according to claim 10, whereinthe circuitry is further configured to generate the first avatar basedon the set of body parameters of the first user and the user profile ofthe first user.
 13. The electronic device according to claim 1, whereinthe circuitry is further configured to: receive sensor data associatedwith biological information of a body of the first user, and determinethe real-time feedback based on the received sensor data.
 14. Theelectronic device according to claim 1, wherein the circuitry is furtherconfigured to: receive a second set of images of a set of users engagedin the first fitness activity, wherein the set of users comprises thefirst user and a second user; generate the augmented-reality displaythat includes a set of avatars associated with the set of users, whereinthe set of avatars comprises the first avatar and a second avatar;determine the posture information associated with each user of the setof users based on the second set of images and the first fitnessactivity; and determine the real-time feedback for each user of the setof users based on the determined posture information.
 15. The electronicdevice according to claim 1, wherein the circuitry is further configuredto train the first neural network model on one or more features toclassify the posture information of the first user and to determine thereal-time feedback, and the one or more features comprise a set of keypoints of the posture information for each fitness activity of aplurality of fitness activities, a movement of one or more body partsfor each fitness activity, a number of repetitions for each fitnessactivity, the duration of each fitness activity, and a breathing patternassociated with each fitness activity.
 16. A method, comprising:receiving a first set of images of a first user, wherein the first setof images is captured for a duration in which the first user is engagedin a fitness activity; generating an augmented-reality display thatincludes a first avatar and an image of the first user based on thefirst set of images; controlling a display device to render thegenerated augmented-reality display, wherein the renderedaugmented-reality display includes the first avatar configured toperform the fitness activity; determining posture information of thefirst user based on the first set of images for the duration in whichthe first user is engaged in the fitness activity; determining real-timefeedback based on application of a first neural network model on thedetermined posture information, wherein the determination of thereal-time feedback is in response to performance of the fitness activityby the first user; and controlling the first avatar to output thedetermined real-time feedback in the augmented-reality display.
 17. Themethod according to claim 16, further comprising applying a secondneural network on the first set of images to determine the postureinformation of the first user for the duration of the fitness activity,and the posture information includes a plurality of key pointscorresponding to joints of a body of the first user.
 18. The methodaccording to claim 16, wherein the determined real-time feedback isassociated with at least one of a movement of one or more parts of abody of the first user, a posture of the first user, a number ofrepetitions of the fitness activity, the duration of the fitnessactivity, or a breathing pattern of the first user during the fitnessactivity.
 19. The method according to claim 16, further comprising:receiving a second set of images of a set of users engaged in thefitness activity, wherein the set of users comprises the first user anda second user; generating the augmented-reality display that includes aset of avatars associated with the set of users, wherein the set ofavatars comprises the first avatar and a second avatar; determining theposture information associated with each user of the set of users basedon the second set of images and the fitness activity; and determiningthe real-time feedback for each user of the set of users based on thedetermined posture information.
 20. A non-transitory computer-readablemedium having stored thereon, computer-executable instructions that whenexecuted by a processor of an electronic device, causes the processor toexecute operations, the operations comprising: receiving a first set ofimages of a first user, wherein the first set of images is captured fora duration in which the first user is engaged in a fitness activity;generating an augmented-reality display that includes a first avatar andan image of the first user based on the first set of images; controllinga display device to render the generated augmented-reality display,wherein the rendered augmented-reality display includes the first avatarconfigured to perform the fitness activity; determining postureinformation of the first user based on the first set of images for theduration in which the first user is engaged in the fitness activity;determining real-time feedback based on application of a first neuralnetwork model on the determined posture information, wherein thedetermination of the real-time feedback is in response to performance ofthe fitness activity by the first user; and controlling the first avatarto output the determined real-time feedback in the augmented-realitydisplay.